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It is wellknown that the majority of South African learners achieve extremely poorly in Mathematics. Many claim that one 
of the causes of this poor attainment is teachers’ weak knowledge of mathematics, and propose that improving teachers’ 
mathematical knowledge would improve learner attainment. However, the evidence-base for this proposed solution is 
currently relatively weak. We report on a quasi-experimental study examining the learning gains of Grade 10 learners from 
five secondary schools in the Johannesburg area, whose teachers participated in a year-long professional development course 
aimed at improving the teachers’ knowledge of mathematics for teaching. Statistical analyses of pre- and post-test results 
show that the intervention group of learners (N = 586) taught by teachers who had participated in the professional 
development (N = 14) outperformed a matched control group of learners (N = 217) taught by teachers in the same schools (N 
= 7). An effect size of d = 0.17 for the intervention group is equivalent to two months’ additional progress. While the 
learning gains are small, they are statistically significant. These findings provide empirical support for claims that attending 
to teachers’ mathematical knowledge can impact learners’ attainment. Suggestions are made regarding the form and 
substance of such professional development. 
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Introduction 

Across the world there are attempts to improve teachers’ mathematical knowledge in order to raise learner 
attainment. In South Africa, despite many years of mathematics professional development programmes aimed at 
redressing the devastating effects of apartheid schooling and apartheid teacher education, there is little evidence 
to show we have made much progress at the level of the learner. Claims of lack of impact are typically based on 
results of summative national and international assessments, such as the Trends in International Mathematics 
and Science Study (TIMSS), Programme for International Student Assessment (PISA), Southern Africa 
Consortium for Monitoring Education Quality (SACMEQ), the Annual National Assessments (ANA), and the 
National Senior Certificate (NSC) exams. This lack of impact is often attributed, at least in part, to teachers’ 
poor mathematical knowledge (Carnoy, Chisholm, Addy, Arends, Baloyi, Irving, Raab, Reeves, Sapire & Sorto, 
2011; Taylor, N & Taylor, S 2012) and there are instances where it has been shown that teachers don’t know 
well enough the mathematics their learners need to learn (Bansilal, Brijlall & Mkhwanazi, 2014; Carnoy et al., 
2011; Taylor, N & Taylor, S 2012). 

A central goal of the Wits Maths Connect Secondary Project 1 (WMCS) is to develop models of pro¬ 
fessional development for secondary mathematics teachers that strengthen teachers’ relationship to math¬ 
ematics, and that ultimately lead to learning gains at all levels of secondary schooling. This requires deliberate 
attention to conceptualising, designing and implementing a professional development programme and then 
researching its impact on learner attainment. The process of researching a development initiative such as this is 
dependent on a carefully-conceptualised research design and the collection of robust evidence. In this paper we 
report on the initial stages of such a programme of research and development, and the evidence collected thus 
far for the impact of the professional development courses on learning gains. 

A key decision in the research design is to reconsider how the impact of teachers’ knowledge on learner 
attainment is measured. Learner results on national assessments are typically reported in terms of pass rates 
(exempli gratia (e.g.) NSC results) or in terms of average marks (e.g. ANAs). Yet in a context where the 
majority of learners are mathematically under-prepared for their current grade, pass rates and average marks are 
not appropriate measures by means of which to investigate change. We propose that, given the current education 
context in the country, learning gains is a more robust measure of change in learner performance, particularly 
when seeking to investigate links between teacher knowledge and learner attainment. In this study, we refer to 
‘learning gains’ as changes in learners’ scores in a pre-test/post-test design over one academic year. In this way, 
we are able, to some extent, to attribute learning gains to the teaching that learners receive in that year. 

There is little evidence that mathematics professional development programmes in South Africa are having 
an impact on learner attainment. Furthermore, there are not yet adequate frameworks for evaluating the impact 
of professional development in the country. We investigated whether the professional development courses 
offered by WMCS constituted an intervention worth pursuing. We thus sought indicative rather than conclusive 
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results to make a case for the continuation (or 
alternatively the termination) of the professional 
development courses. 

The results reported here show that learners 
taught by teachers who participated in the pro¬ 
fessional development programme outperformed 
learners taught by teachers who did not participate 
in the programme. These indicative results suggest 
that the courses offered by WMCS are worth 
pursuing further, but that a more rigorous investi¬ 
gation into their impact on learner attainment is an 
essential future step. In addition, a secondary 
objective was to show that more rigorous evalu¬ 
ations of educational interventions using quasi- 
experimental designs are possible in the context of 
South African schools. 

Teacher Knowledge and Learner Attainment 
Following Shulman and colleagues’ initial concep¬ 
tualisation of teacher knowledge (Shulman, 1986, 
1987), a great deal of work has been done on 
teachers’ mathematical knowledge for teaching. 
There is widespread agreement that the knowledge 
teachers require for teaching Mathematics is more 
than sound content knowledge of mathematics 
itself. While some (e.g. Krauss, Baumert & Blum, 
2008) refer to this additional knowledge as peda¬ 
gogical content knowledge (PCK) following Shul¬ 
man, others (e.g. Ball, Thames & Phelps, 2008) 
have attempted to disaggregate both content know¬ 
ledge (CK), or subject matter knowledge (SMK), 
and PCK further. However, there is some lack of 
clarity about the boundaries between CK/SMK and 
PCK. So, while we find the two terms useful for 
emphasising different aspects of teacher know¬ 
ledge, we argue that they are problematic when 
used as analytical constructs. We believe it is more 
productive to consider an amalgam of mathematical 
and teaching knowledge, and so we use the term 
“mathematics-for-teaching” (MfT) (Adler, 2005; 
Adler & Davis, 2006) to encompass both subject 
content knowledge and mathematics-specific peda¬ 
gogical knowledge. 

Until the mid-eighties, teachers’ subject 
knowledge was only measured indirectly, and 
through proxy measures, such as state certification, 
number of Mathematics or Mathematics Education 
courses taken, and years of teaching Mathematics 
(Even, 1993). While it has been argued that such 
proxy measures are neither good measures of 
teachers’ knowledge (Ball, Bass & Hill, 2004), nor 
good predictors of learner attainment (e.g. Hill, 
Ball & Schilling, 2008), there is some evidence of 
their predictive power in secondary Mathematics. 
For example, Darling-Hammond (2000) and 
Goldhaber and Brewer (2000) found a positive 
relationship between state certification and learning 
gains in the United States (US). Monk (1994) 
found a positive relationship between number of 


Mathematics courses taken and student achieve¬ 
ment, although the effects were very small. 

We suggest these proxy measures may hold 
some applicability for contexts such as South 
Africa, where teachers’ mathematical knowledge 
bases are generally poor, and where, based on 
anecdotal evidence from our project schools, too 
many teachers who are teaching Mathematics at 
lower secondary level, have little, if any, training as 
Mathematics teachers. In such cases the number of 
post-school Mathematics courses taken does 
matter, and may be a predictor, albeit a poor one, of 
learner attainment. So, while Hill, Rowan and Ball 
(2005) argue that some research findings on learn¬ 
ing gains from the Global South (e.g. Harbison & 
Hanushek, 1992, Mullens, Murnane & Willett, 
1996) may not generalise to the US, we likewise 
suggest that the dismissive stance of some in the 
Global North to proxy measures may be in¬ 
appropriate in the Global South. That said, we 
agree that proxy measures alone are insufficient as 
measures of teachers’ mathematical knowledge. 

Attempts to make use of more direct measures 
of teacher knowledge have taken different forms. 
Some have tested teachers on the same/similar 
level content as their learners. For example, Har¬ 
bison and Hanushek (1992) administered the same 
test to Grade Four learners and their teachers in 
rural Brazil, and found that teachers’ scores were a 
strong predictor of learners’ scores. Working in 
Belize, Mullens et al. (1996) found that teachers’ 
scores on the national primary school-leaving 
examination for Mathematics were a good predictor 
of the Mathematics scores of their Grade Three 
learners. In South Africa, the SACMEQ III study 
was extended to include testing of Grade Six 
teachers’ mathematical knowledge on items typical 
of Grade Six level (and lower), where 15 items 
were common to both the teacher and learner tests. 
Taylor, N and Taylor, S (2012) report that teachers 
and learners performed well on only two simple 
items but that both teachers and learners performed 
poorly on eight items. This suggests that Grade Six 
teachers do not know much of the mathematics 
they teach well enough to teach it. 

In the US and Germany, large research pro¬ 
jects have developed sophisticated measures that 
attempt to disaggregate different components of 
teacher knowledge (Hill et al., 2008, Krauss et al., 
2008). In the Study of Instructional Improvement, 
Hill et al. (2005) found that the mathematical- 
knowledge-for-teaching of Grade One and Grade 
Three teachers was a stronger predictor of learner 
attainment than were proxy measures, such as 
number of courses taken in Mathematics or 
Mathematics Methodology, or years of teaching 
experience, or average daily length of maths les¬ 
sons. In Germany, Baumert, Kunter, Blum, Brun¬ 
ner, Voss, Jordan, Klusmann, Krauss, Neubrand 
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and Tsai (2010) found that teachers’ scores on 
SMK 11 and PCK items were strongly linked to the 
mathematics teacher preparation they had received, 
with those preparing to teach at higher academic 
levels in schools outperforming their counterparts 
on both SMK and PCK. They argue that teachers’ 
PCK is a better predictor of learner progress than 
SMK. While both studies found associations 
between teacher knowledge and learner attainment, 
neither study investigated the way in which 
interventions on teacher knowledge impact learner 
attainment. 

Teachers’ knowledge matters in all learning 
contexts. However, it matters more in contexts of 
poverty and low achievement. Nye, Konstant- 
opoulos and Hedges (2004) found variance in 
learning gains attributable to teaching to be higher 
in low socio-economic status (SES) schools, and 
Krauss et al. (2008) have shown that differences in 
teachers’ PCK had a larger impact in low-SES and 
low-achievement contexts in Germany. 

Professional Development and Learner Attainment 
The problem of how professional development can 
be designed to improve learners’ attainment is not 
confined to developing countries like South Africa. 
A decade ago, a survey of the international 
literature found the field to be dominated by small- 
scale qualitative studies (Adler, Ball, Krainer, Lin 
& Novotna, 2005). More recently, in a review of 
the literature Gersten, Taylor, Keys, Rolfhus and 
Newman-Gonchar (2014) identified 643 studies of 
professional development relating to school Math¬ 
ematics. Of these, only five met the ‘What Works 
Clearinghouse’ evidence standards. Of these five, 
only two found positive effects on learners’ attain¬ 
ment, and only one of these five studies (Sample 
McMeeking, Orsi & Cobb, 2012) reported the ef¬ 
fects of an intervention focused on teachers’ math¬ 
ematical knowledge. Sample McMeeking et al. 
(2012) report the effects of a study in which middle 
school teachers 111 in the US participated in one or 
two university summer courses in Mathematics, 
lasting two to three weeks. The courses consisted 
of 80% mathematics content and 20% mathematics 
pedagogy. They found a discernible effect size on 
learner attainment for those teachers who had 
attended two courses, but not for those who had at¬ 
tended only one course. This effect size is reported 
by Gersten et al. (2014) as 0.20. lv 

We move now to describe the content, struc¬ 
ture and approaches of the WMCS professional 
development courses, and thus to describe what 
MfT was offered to teachers. 

The Transition Maths Intervention 
Most mathematics professional development pro¬ 
grammes in South Africa can be described as 
taking either a repair approach or a conceptual 
approach to the mathematics in their offerings. 


Repair approaches focus on teachers redoing school 
Mathematics in the same ways as their learners 
would learn it. Here, teachers rehearse the steps 
necessary to solve typical tasks from the school 
curriculum. Conceptual approaches frequently 
work from the assumption that teachers’ math¬ 
ematical knowledge is procedural, and thus in¬ 
adequate, and that interventions should provide 
them with a deep conceptual understanding to 
complement their procedural knowledge. Both 
approaches have limitations. A repair approach 
tends to position teachers as school learners, which 
stands in stark contrast to generally-held principles 
of professional development (e.g. Clarke, 1994), 
and does not go beyond a narrow knowledge of the 
mathematics of the curriculum to address math¬ 
ematics for teaching more broadly. Conceptual 
approaches focus extensively on developing con¬ 
ceptual insight, often through extended problem¬ 
solving tasks. While we value conceptual insight 
and challenging tasks, our concern is that often 
such programmes adopt an exclusively conceptual 
approach with little regard for the role of pro¬ 
cedures or procedural fluency (Kilpatrick, Swafford 
& Findell, 2001) in mathematical proficiency, and 
the place of procedures in typical tasks in 
secondary school Mathematics. Much of school 
Mathematics is characterised by applying familiar 
procedures, and it is thus important to deal with 
such features of school Mathematics in professional 
development, and to do so in ways that are princi¬ 
pled, and thus constructive for teachers and learn¬ 
ers. 

The Transition Maths (TM) courses form the 
backbone of the professional development work of 
WMCS, and were designed with the assumption 
that focusing on teachers’ MfT leads to better 
teaching, which ultimately translates into increased 
learner attainment. We thus assume a direct effect 
on teacher knowledge, and an indirect and delayed 
effect on learner attainment. The courses focus on 
mathematics content (75%) and aspects of math¬ 
ematics teaching (25%), and thus are structured in a 
similar ratio to Sample McMeeking et al.’s (2012) 
programme. Each course comprises eight two-day 
contact sessions over a year, with independent 
work between these sessions, which includes tutori¬ 
als on the mathematics content, and tasks related to 
teaching. 

While the courses have distinct foci, both 
focus on learning MfT through revisiting known 
mathematics and learning new mathematics (Pour- 
nara, 2013). The goals of revisiting are to deepen 
teachers’ grasp of the content, frequently by 
exploring extreme cases or by problematising as¬ 
pects that may be taken for granted rather than 
redoing to improve procedural fluency. Revisiting 
builds on, strengthens and extends teachers’ 
existing knowledge of the mathematics at hand. 
Whilst revisiting tasks are structured around 
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‘known’ mathematics, the activity focuses on is¬ 
sues such as making connections between different 
representations, and between different sections of 
the curriculum. 

In TM1, we revisit curriculum content of 
Grades Eight to 10, and for these teachers we con¬ 
sider the content of Grades 11 and 12 as ‘new 
maths’. In TM2 we revisit the content of Grades 11 
and 12, and then extend this beyond the school cur¬ 
riculum to some aspects of tertiary mathematics. In 
both courses we treat new content in the school 
curriculum as new mathematics. We show the 
distinction between revisiting known mathematics 
and learning new mathematics through the topic of 
functions, a key concept in both the school curricu¬ 
lum and advanced mathematics. 

In TM1, we begin with a process orientation 
to function (Sfard, 1991), emphasising the catch- 
phrase “graphs come from points, and points come 
from the relationship between inputs and outputs”. 
We then extend to a structural view of function 
(Sfard, 1991), with a focus on transformations of 
functions. These approaches are reinforced by 
working with multiple representations of functions. 
We begin with the familiar (to teachers) linear and 
basic quadratic function, and extend to other 
functions in the Further Education and Training 
(FET) curriculum and beyond. In TM2, we build 
further on a structural view of function as we 
extend to more advanced functions, including 
inverses, and beyond the FET curriculum with 
work on algebra of functions, and piecewise 
functions. In both courses we work with the square- 
root function (f(x) = a\lx — b + c), which is not 
included in the school curriculum, but which 
exemplifies key aspects of function, such as 
domain, range and inverse in powerful ways. 

In both courses, we pay attention to the key 
procedures that learners are required to learn, such 
as factorisation, solving equations and proving 
riders. Our emphasis is on studying the routine as a 
set of logically derived steps, rather than a cue- 
based exercise in manipulating symbols. 

With regard to aspects of teaching, we 
structure our intervention around the notion of 
mathematical discourse in instruction (Adler & 
Ronda, In press; Adler & Venkat, 2014). We 
operationalise this through a focus on aspects that 
are typical of teaching, irrespective of pedagogy, 
videlicet (viz.) choosing and using examples; 
providing explanations and justifications; and 
learner activity. A key strength of this approach is 
that it focuses on issues that are sufficiently close 
to teachers’ current practice as to be possible to 
implement. We work from the assumption that 
better teaching is characterised by more thoughtful 
selections of examples and tasks, and by math¬ 
ematical explanations that focus explicitly on the 
mathematical object (e.g. concept or procedure) 
that the teacher intends the learners to learn. This is 


achieved through attention to the appropriate use of 
mathematical terminology, by means of a range of 
relevant representations. We focus on opportunities 
for learner participation that go beyond single-word 
responses, completing teachers’ sentences, and 
copying procedures from the chalkboard. We work 
on these three components with teachers in the TM 
courses, examining records of practice using these 
notions. 

Methods 

We used a quasi-experimental design to assess the 
effect of the TM intervention on learner attainment. 
In this section, we describe the sampling, design 
and content of the test, and the analytic methods 
that were used. 

Sample 

We worked in 11 secondary schools in the greater 
Johannesburg area, with six schools located in 
townships and four in suburban areas. The townhip 
schools are no-fee schools, while those in the 
suburbs may be described as low-fee schools, 
although they generally struggle to collect these 
fees. Most of the schools have been classified as 
“under-performing” or “priority schools” at some 
stage in the life of the project. Consequently, they 
have been subjected to increased bureaucratic 
control from the provincial education department, 
which includes the requirement that they write 
externally-set examinations twice a year. Over the 
duration of the project, the Mathematics pass rates 
in some schools have fluctuated considerably. 
There have been substantial demographic shifts in 
recent years in the suburban schools, and so the 
vast majority of learners in these schools are black. v 

The test was conducted in five project schools 
during the 2013 academic year. The selection of 
schools was purposeful, to include fee-paying and 
non-fee-paying schools, as well as those where 
teachers teaching Grade 10 Mathematics in 2013 
included TM participants and non-TM participants. 
In addition, based on our previous experiences of 
the challenges in collecting learner test data in the 
project schools, it was important to select schools 
where teachers were committed to running the pre- 
and post-tests, and thus supporting our research. 

Twenty-one teachers participated in the study, 
14 of whom were TM teachers, while seven were 
non-TM teachers. The selection of all teachers was 
based on their teaching of at least one Grade 10 
Maths class in one of the five selected schools and 
their willingness to participate in the study. We 
were fortunate that all Grade 10 Maths teachers in 
the five schools agreed to participate in the study. It 
is important to acknowledge here that the pro¬ 
vincial department offered catch up lessons for 
learners, and workshops for teachers at various 
points in time, and further, that teachers partici¬ 
pated in various activities related to their profes- 



South African Journal of Education, Volume 35, Number 3, August 2015 


5 


sional lives. With respect to learners, all learners 
across all schools had opportunity to attend catch¬ 
up lessons. With respect to teachers, we cannot 
consider the WMCS intervention as somehow 
divorced from this wider contact, where some TM 
teachers and non-TM teachers in different schools 
might participate in teacher support activities. 
However, it was clear from our interactions with 
school leadership that the WMCS professional 
development was considered to be the dominant 
initiative amongst Maths teachers in the school. 

The timing of teachers’ participation in the 
courses is important in relation to the timing of the 
pre- and post-tests. Six teachers completed the 
TM1 course in 2012, whilst another four teachers 
enrolled for the TM1 course at the beginning of 
2013, and completed it at the end of 2013. Four 
teachers participated in the TM2 course, which ran 
from July 2012 to June 2013. Thus, there were 
eight teachers enrolled in a course for at least part 
of the data collection period. This is worth noting 
because research (e.g. Clarke, 1994) suggests that 
the impact of professional development pro¬ 
grammes on teachers’ classroom mathematics 
practice is delayed. 

The pre-test was written by 882 learners, 
while only 803 learners wrote both pre- and post¬ 
test. We analysed only the scripts of those learners 
who had written both tests in order to compare 
learning gains. In total, 586 learners (73%) were 
taught by TM teachers, with 217 (27%) learners in 
the control group, taught by non-TM teachers." We 
refer to those taught by TM teachers as TM 
learners, and to the control group as non-TM 
learners. The breakdown across TM1 and TM2 is 
shown in Table 1. 

Table 1 Numbers of learners in each Transition 
Maths group and the control group, who 
wrote pre-test and post-test. 


N 

TM1 

392 

TM2 

194 

Control 

111 

TOTAL 

803 


The Grade 10 Learning Gains Test 
The Grade 10 Learning Gains Test was designed by 
the project team as a test of key aspects of the 
Grade 10 algebra, functions and geometry curri¬ 
culum, using typical curriculum items. The total 
mark for the test was 76. Algebra content (51% of 
marks) included simplification, substitution, 
factorisation, equations and linear patterns. Func¬ 
tions content (36%) included function notation, 
properties of linear functions, quadratic functions 
and related transformations. Particular emphasis 
was placed on moving between different 
representations of functions. Geometry (13%) dealt 
with triangles, quadrilaterals, and congruency, with 
one question requiring formal proof. A selection of 


‘look-alike’ questions is provided in Appendix A. 
The look-alikes are necessary to ensure 
confidentiality of the test items. They are questions 
that are close, but not identical, to those given in 
the actual test. For example, if a test item appeared 
as “factorise fully: am 2 — 3am” then a look-alike 
might appear as: “factorise fully: kp 3 — 2 kp”. 

The test was designed to contain items with a 
range of difficulty. For example, sample questions 
1, 4a and 5 would be considered easy questions at 
Grade 10 level, since they deal with Grade Nine 
content. Questions 3, 4b, 4c and 4d tested typical 
content introduced in Grade 10. Question 2 is a 
relatively difficult example of a quadratic equation 
at Grade 10 level. Items were revised through an 
iterative process so as to reduce the complexity of 
easier items, so that, for example, fewer con¬ 
cepts/procedures were tested in a single question. 

The test was designed to be administered 
during a typical maths lesson (approximately one 
hour), in order to reduce interference in the 
teaching schedule. 

A Rasch analysis was used to assess the 
validity of the test (Hodgen, Pillay, Adler & 
Pournara, 2014). In brief, this analysis showed that 
the test was fit for the purpose of comparing the 
learning gains between the TM groups and the 
control group of learners. The test performed well 
on dimensionality tests. Almost all items provided 
an excellent fit to the Rasch model, with occasional 
misfit well below the level that would degrade 
measurement. However, there were relatively few 
easier items, leading to poorer discrimination 
amongst the lowest attaining learners. As a result, 
the test may underestimate the gains made by the 
lower attaining learners. 

Analysis 

In comparing the overall changes in mean scores, 
we used both descriptive and inferential statistics 
for both TM groups, together and separately. We 
report comparisons using overall changes in the 
mean score. Using SPSS 22.0, regression was used 
to compare differences between learners in classes 
taught by TM teachers and the control. We report 
the TM group as a whole, together with the 
comparison of each TM group to the control. In 
order to calculate a meaningful effect size, we 
calculated Cohen’s d, and then interpreted this 
value following Higgins, Kokotsaki and Coe 
(2012) in terms of additional months of progress, 
beyond the progress that might be expected of 
learners without the intervention. 

Results 

In Figure 1 we present a graphical comparison of 
the TM groups with the control. For the TM groups 
as a whole, and for TM1, the TM learners’ initial 
attainment was below that of the control, whereas 
their attainment after a year’s teaching by the TM 
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teachers was above that of the control. The 
attainment of TM2 learners on the pre-test was 
below that of the control and, while the gap in 
attainment narrows over time, their attainment in 
the post-test was still slightly below the control 
group. 

Table 2 shows pre- and post-test results for 
the TM group as a whole. The TM group gains are 


greater than those of the control, although the 
variation of the scores and gains is considerable. In 
Table 3, we disaggregate the results for TM1 and 
TM2. Both show gains over the control, although 
the gain for TM1 is much larger. However, the 
variation in TM1 gains is quite large, which 
suggests that some learners, or some TM1- 
teachers’ classes, did better than others. 




Figure 1 Mean pre- and post-test scores for the TM groups as a whole and for each of TM1 and TM2 separately 


Table 2 Mean pre- and post-test scores, standard deviations (SD) and gains for the TM groups as a whole 




Pre-test score 

Post-test score 

Gain 


N 

Mean 

SD 

Mean 

SD 

Mean SD 

TM1 and TM2 

586 

4.49 

3.40 

7.71 

5.12 

3.22 3.78 

Control 

217 

4.95 

3.63 

7.49 

5.01 

2.54 3.28 


Table 3 Mean pre- and post-test scores, and gains for the TM1 and TM2 classes separately 



N 

Pre-test score 

Post-test score 

Gain 

Mean 

SD 

Mean 

SD 

Mean 

SD 

TM1 

392 

4.49 

3.35 

7.89 

5.32 

3.40 

4.03 

TM2 

194 

4.48 

3.51 

7.34 

4.70 

2.86 

3.20 

Control 

217 

4.95 

3.63 

7.49 

5.01 

2.54 

3.28 


It is important to note that both groups start 
from a very low base and that the gains for both 
groups are relatively small. For example, the 
average pre-test percentage score for the combined 
TM group was 5.9%, and this improved to 10.1% 
on the post-test. This indicates that the majority of 
learners had not grasped the content of Grade 10 
Mathematics in algebra, function and geometry. 
We discuss this further in the conclusion. 

The gains for the TM group were significantly 
greater than the control for the TM group as a 
whole (mean difference, 0.68; SE, 0.29; t (801) = 
2.33; p = 0.020).™ When considered separately, the 


gains for the TM1 group were also significant 
(mean difference, 0.86; SE , 0.32; t (607) = 2.68; p 
= 0.008). However, the gains for the TM2 group 
were not significantly greater than the control 
group, although the gains were positive (mean 
difference, 0.31; SE, 0.32; t (409) = 0.97; p = 
0.331). 

In Table 4, we show the results of a linear 
regression for the entire sample, with post-test 
scores as the dependent variable. In this analysis, 
pre-test scores are treated as an independent 
variable in order to control for prior attainment. 
Learner participation in TM was treated as a 
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dummy independent variable. In Table 5, we show 
similar regressions for the TM1 and TM2 learners, 
considered separately. As would be expected, the 
results are similar to the f-tests reported above. In 
each case, the raw effect (the unstandardised TM 
coefficient) is slightly greater than the mean 
differences reported above, although in practical 
terms, the effects are very similar. It can be seen 
from the standardised coefficients that the effect of 


participation in a TM class is small in comparison 
to prior attainment (as measured by pre-test scores). 

In Table 6, we show the effect sizes for the 
TM group as a whole, and for TM1 and TM2 
separately. Whilst all the effects are positive, the 
greatest gains were made by the TM1 group, where 
the effect size of 0.21 is equivalent to three months’ 
additional progress (Higgins et al., 2012). 


Table 4 Summary of regression analysis for the entire sample of students in comparison to the control. 

Dependent variable: post-test score. Independent variables: ‘was learner in a TM class?’ and pre-test 
score. Standard error for unstandardised coefficients shown in brackets. 



Unstandardised coefficients 

Standardised coefficients 


Intervention group 

Was learner in a TM class? 

Pre-test score 

Teacher did TM course? 

Pre-test score 



B 

B 

P 

P 

P 

Both TM groups 

0.688 

1.024 

0.060 

0.698 

0.018 


(0.291) 

(0.037) 





Table 5 Summary of separate regression analyses for the TM1 and TM2 in comparison to the control. 

Dependent variable: post-test score. Independent variables: ‘was learner in a TM class?’ and pre-test 
score. Standard error for unstandardised coefficients shown in brackets. 



Unstandardised coefficients 

Standardised coefficients 


Intervention group 

Was learner in a TM class? 

Pre-test score 

Teacher did TM course? 

Pre-test score 



B 

B 

P 

P 

P 

TM1 

0.874 

(0.321) 

1.038 

(0.044) 

0.080 

0.689 

0.007 

TM2 

0.320 

(0.321) 

1.017 

(0.045) 

0.033 

0.747 

0.320 


Table 6 Gains, pooled standard deviations and effect sizes ( d) for the TM-classes as a whole and for TM1 and 


TM2 separately™ 1 



Gain 

Pooled SD 

d 

Months’ progress 

Description 

All TM 

0.627 

3.653 

0.17 

2 

Low 

TM1 

0.803 

3.783 

0.21 

3 

Medium 

TM2 

0.285 

3.242 

0.08 

1 

Low 


Discussion 

These results indicate a small but statistically 
significant effect of d= 0.17 on learning gains for 
the TM intervention as a whole. According to 
Higgins et al. (2012), this is a small effect size, and 
equivalent to a gain of two months’ additional pro¬ 
gress. Although the effect is small, it is an indirect 
effect, which indicates that the TM intervention had 
an effect on learner attainment, even though the 
intervention was principally directed at increasing 
teacher knowledge, rather than learner attainment. 

The effect for the TM1 intervention compared 
to the control is a medium effect of d = 0.21, which 
is a medium effect size, and equivalent to a gain of 
three months additional progress (Higgins et al., 
2012). 

On the other hand, the TM2 intervention, 
which was directed at more advanced mathematical 
content knowledge, shows a much smaller effect 
that is not statistically significant. However, this 
shows that the TM2’s focus on advanced teacher 
knowledge beyond the Grade 10 curriculum has a 
positive, if modest, effect on learner attainment 
equivalent to around one month. 


The differences in the gains between the 
TM1-group and the TM2-group are not easily 
accounted for, and given the small number of TM2 
teachers in the sample, any suggestions must be 
made with extreme caution. One possible reason is 
that the TM2 course did not pay much attention to 
Grade 10 mathematical content, nor to the teaching 
of this content. By contrast, in TM1, there was a 
great deal of attention to Grade 10 content, and 
some attention to the associated teaching issues. 

It is important to note a number of limitations 
to these results, in particular that the sample of 
teachers was small (N = 21), and that the variation 
in the gains made by learners was large, where 
some learners made much larger gains than others, 
even within the same class. In addition, the control 
group were from similar classes in the same 
schools as the intervention group, and we have 
controlled for prior attainment in our analysis. 
However, whilst we are reasonably confident that 
the intervention and control classes are similar, this 
does not constitute a rigorously matched sample. 
Hence, the results should be treated as indicative, 
rather than conclusive. Furthermore, we did not 
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gather data on how teachers taught. We assume that 
non-TM teachers taught in similar ways to their 
previous teaching. While this is a reasonable 
assumption, the inability to confirm this 
nevertheless constitutes a potential limitation of the 
study. 

Conclusion 

In this paper, we have reported the impact of a 
professional development programme on learners’ 
attainment in mathematics. While we treat our re¬ 
sults as indicative rather than conclusive, this study 
makes several important contributions. Firstly, we 
have provided evidence that working on teachers’ 
MfT through the TM courses led to learning gains 
amongst Grade 10 learners. While these gains were 
small, the effect size is equivalent to two months of 
additional progress, and similar in magnitude to 
that reported by Sample McMeeking et al. (2012). 
This gives us confidence to pursue the model of 
professional development linked to the TM 
courses. Our confidence is boosted by the fact that 
the effects of professional development on learning 
gains are always secondary effects, and tend to lag 
behind the completion of teachers’ participation in 
professional development. It is therefore particu¬ 
larly encouraging that we have obtained effects so 
soon after teachers’ participation in the TM 
courses. Secondly, we have demonstrated that the 
notion of learning gains is a more productive way 
of investigating improvements in the system than 
comparing results, across years, of summative or 
one-off assessments such as TIMSS, NSC or the 
ANA. Thirdly, the methodology of the study pro¬ 
vides a productive means by which to associate 
learning with teaching in ways that have not yet 
been attempted in mathematics education in South 
Africa. Since previous studies of learner attainment 
have been divorced from specific teachers, it has 
not been possible to explore direct links between 
learning and teaching. Furthermore, our study 
shows the potential for rigorous evaluation of 
professional development interventions in the 
South African context. This, too, gives us confi¬ 
dence to pursue this line of research further. 

Nevertheless, we note two issues that must be 
taken into account in further research. The first is a 
caveat, and relates to the teacher sample. It is 
possible that teachers who chose to participate in 
the TM courses were more motivated than their 
colleagues who chose not to participate, and this 
may lead to bias in the sample. In future research it 
will be important to attempt to set up randomised 
teacher samples. This is no easy task, partly 
because the need for a control group is key for 
purposes of comparison, and also because having a 
control group within the school to some extent 
controls for contextual factors that impact teachers’ 
work. The second issue concerns the low levels of 


performance in both the pre- and post-tests, which 
show that, despite the improvements, the majority 
of learners in the study were not adequately pre¬ 
pared for Grade 10 Mathematics, and that by the 
end of their Grade 10 year, were not adequately 
prepared for Grade 11 Mathematics. This should 
not detract from the improvements made by TM 
teachers, but it does reflect the low base from 
which teachers are required to work. It is important 
to acknowledge the additional demands this places 
on teachers, and future research needs to investi¬ 
gate how professional development programmes 
might support teachers in addressing learners’ 
under-preparedness in mathematics. 
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Notes 

i. The Wits Maths Connect Secondary Project is one of 
the FirstRand Foundation-National Research 
Foundation Mathematics Chair projects, led by Jill 
Adler. 

ii. In this study the authors refer to CK rather than 
SMK. 

iii. Middle school in the US caters for learners aged 
between 10 and 14 years old. 

iv. Gersten et al. (2014) report effect sizes using 
Hedges’s g, a similar measure to Cohen’s d used later 
in this paper, but corrected for bias. 

v. We use the term “black” as a generic term for the 
apartheid race classifications to include African, 
Indian and Coloured. 

vi. As is commonplace in any intervention, we 
experienced learner “drop-out” from post-test and 
teacher movement during the year, but this was 
relatively small. Across the schools there were also 
three new teachers who were allocated Grade 10 
Mathematics during the course of the year. 

vii. The Levene's test suggests that the variances for both 
TM groups together and for the TM1 group alone are 
different to the control. A “variances not assumed” t- 
test indicated lower p values of 0.013 (All TM) and 
0.05 (TM1). In the paper we report the slightly more 
conservative result of the standard f-test. A non- 
parametric test (Mann-Whitney U) was also 
conducted in both these cases. This showed a 
significant p value of 0.022 (All TM) and 0.014 
(TM1). 

viii. These effect sizes are calculated using the slightly 
more conservative mean differences rather than 
regression coefficients. 
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1) Factorise fully: —4 k + 20 

2) Solve for the unknown: x(x + 3) = 10 


3) Given f(x) — 6 — x, determine x if fix) — 14 


4) 


The diagram shows the graphs of 
y = x + 2 and y = x 2 — 4 

a) Write down the coordinates of B. 

b) Write down the minimum value of the parabola. 

c) The 2 graphs intersect at A and F. Determine the coordinates of 
A and F. 

d) Assume the graph of the parabola is translated 2 units down. 
Give the equation of the new graph. 




5) Determine the size of x. Show how you obtained your answer. 
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